Dagster Orchestrated Pipeline

In this data engineering pipeline project, I leveraged Dagster as the orchestrator to streamline the end-to-end data processing workflow. The pipeline begins by extracting data from GitHub repositories, utilizing its API to gather relevant information. Next, the extracted data undergoes transformation processes tailored to the specific project requirements. Finally, the transformed data is visualized using GitHub Gists, enabling clear and concise representations of the insights derived from the data. This project showcases the power of Dagster as a robust orchestrator, combined with the integration of GitHub for data extraction and visualization, ultimately providing valuable and visually appealing insights.